The correlation matrix indicates that chest pain type (cp) and maximum heart rate achieved (thalach) have the strongest positive correlations with target, making them critical predictors of heart disease. Conversely, oldpeak (ST depression), number of major vessels (ca) and exercise-induced angina (exang) show strong negative correlations with target, highlighting their importance in predicting heart disease absence. There are no signs of multicollinearity among the independent variables, as no correlations between predictors exceed the threshold of ±0.6, ensuring stable regression models.

For the prediction of the continuous variable value (thalach) we will use linear regression. The results are summarized in this section.

After examining the final model, we can observe some issues in the residual plots that indicate potential concerns with our data. The homogeneity of variance plot shows a curved pattern, suggesting that the residuals are not evenly distributed across the fitted values, meaning that the model may not be capturing the variability in thalach equally for all predictions. The normality of residuals plot indicates some deviation from normality, particularly in the tails, which impact the validity of conclusions.

Additionally, comparing the full and pruned models, we see that removing predictors with high p-values did not significantly improve the fit, as the R² remained relatively low (around 0.39), and the RMSE and MAE values remained nearly the same. This suggests that the linear neither the pruned model may not be the best fit for predicting thalach, and alternative approaches such as tree-based methods may improve predictive accuracy.

After evaluating the data using linear regression, there was potential issues with non-linearity and heteroscedasticity, which affected the model’s predictive performance. To better capture complex relationships between predictors and the target variable, I implemented a lasso regression model with penalty of 0.1 and a lasso tuned regression model with trained/test data.

The analysis of the Lasso regression model highlights the impact of regularization strength on model performance. Excessive penalization (lambda values of 10 or 250) led to over-regularization. By adjusting the penalty to 0.1, the model retained important predictors while still mitigating overfitting. The residual vs. predicted plot indicates that while the model performs reasonably well, there may be some heteroscedasticity, as residuals are not uniformly distributed across predicted values. This suggests that certain ranges of the target variable may be predicted with greater accuracy than others, potentially impacting reliability.

The final lasso tuned model with training/test data gets alike metrics but as the model with λ = 0.1 has higher R² while maintaining similar numbers of RMSE and MAE, it is the better choice. This model balances regularization and predictive power, making it preferable over the more constrained λ = 1 model, which shrinks coefficients more aggressively and slightly reduces prediction accuracy.

After evaluating the data using linear and lasso regression, there was potential issues with heteroscedasticity, which affected the model’s predictive performance. To better capture complex relationships between predictors and the target variable, I implemented a regression tree model and a tuned regression tree model that can automatically detect interactions and split the data into meaningful segments, improving prediction accuracy.

Initially, the first regression tree resulted in improved performance over the linear models. The regression tree model achieved an RMSE of 15.50 and an R² of 0.55, compared to the linear regression and lasso regression model’s RMSE of 17.89 and R² of 0.39. This suggests that the tree model captures more variation in the data than linear regression.

However, to further enhance performance, I tuned the regression tree with training/test split (80% and 20%) by optimizing the cost complexity parameter and tree depth using cross-validation. The tuned regression tree model significantly outperformed the previous models, with an RMSE of 14.56 an MAE of 11.28, and an R² of 0.56. The actual vs. predicted scatter plots show a stronger correlation in the tuned tree, meaning it generalizes better. By refining the model through tuning, we achieved higher predictive power while maintaining interpretability.

For the final model, I will use logistic regression to explore heart disease presence. From the variable importance plot, we can see that males, number of major vessels (ca2, ca3, ca4), chest pain type (cp2, cp3, cp4), and ST depression (oldpeak) are among the most significant predictors in the model. These variables play a critical role in determining heart disease risk, aligning with medical insights that suggest factors like chest pain, blood vessel count, and exercise-induced abnormalities are strong indicators of cardiovascular health. The ROC curve confirms that our logistic model performs well, with an AUC of 0.91, and adjusting the cutoff to 0.74 provides a balance between sensitivity (87%) and specificity (82%).

From the logistic regression equation, we observe that ca5 is a non-significant predictor with a p-value of 0.73. However, as a categorical predictor by nature, I decided to not make modifications as ca5 can be combined with the base category. It simplifies interpretation without losing valuable information.

When predicting the presence of heart disease (target = 1), I coded it so that “Yes” indicates a diagnosis of heart disease, while “No” indicates no heart disease. For this analysis, I performed both classification trees and logistic regression models to compare their predictive performance. Both models in the classification trees achieved a sensitivity of 91%, meaning they correctly identified 91% of individuals with heart disease. Additionally, they demonstrated high specificity (95%) and an overall accuracy of 93%, making it a strong predictive tool. When adjusting the classification threshold, we observed that modifying the cutoff to 0.59 had minimal impact on classification results. This occurred because most predicted probabilities were below 0.59, leading to similar classifications and unchanged performance metrics.The precision is 95% and the AUC is 98%. Overall, both models have a strong predictive performance in detecting heart disease.

I will use all the variables. For this model the cost complexity is set to .001.

The classification tree has 32 leaf nodes, each representing a final decision point in the model. The more splits the tree has, the more complex its decision-making process becomes. This tree structure helps classify individuals based on various health indicators, ultimately predicting whether a person has heart disease or not. Looking at the Variable Importance Plot (VIP), the higher the importance value, the greater the influence of the variable on the model’s decision-making process. In this case, chest pain type (cp) is the most influential predictor, followed by thalassemia status (thal) and maximum heart rate achieved (thalach). These variables significantly impact the likelihood of heart disease, as they are key indicators used in medical assessments.